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(57) Abstract 

A medical system to perform closed chest coronary artery 
bypass graft surgery on the beating heart by using virtual 
target image arrest; components of the system may be: a) 
thoracoscopic 3D video-imaging of the surgical target area 
and stereoscopic video display in operating spectacles; b) 3D 
video-tracking of beacons in the vicinity of the surgical target 
area or tracking by some other means; c) virtual target image 
arrest by real time image manipulation to minimize beacon 
movement; d) target motion compensated thoracoscopic robot 
arms with thoracoscopic surgical tools which track the moving 
target in real time; e) surgeon directed manipulation of robotic 
surgical tools with the surgeon's motions superimposed on 
the automated tracking motions; f) voice command control 
to switch virtual target arrest or automated tracking by robot 
arms on and off; g) voice command control to scale down the 
surgeon* s movements by the robotic tools; h) voice command 
control of zoom module to magnify a specified part in the 
stereoscopic image. 
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1 

Title 



ROBOTIC SYSTEM FOR CLOSE INSPECTION AND REMOTE TREATMENT OF 
MOVING PARTS 

5 

Background of the Invention 

This invention relates to a robotic system for observing and 
10 remote treatment of moving parts. In a preferred embodiment the 
invention relates to a minimally invasive robotic surgical system 
that integrates automated target tracking of a moving body part by 
robotic surgical tools with stereoscopic video-image guided control 
of these tools by the surgeon. Closed chest coronary artery bypass 
15 graft surgery is presented as an example in which such a system is 
utilized to attach the distal end of a bypass graft to a coronary 
artery without the need to arrest the heart. 

In the Western countries, coronary artery disease (narrowing 
20 of the arteries which supply blood to the muscle cells of the 
heart) is the leading cause of death and a major cause of morbid- 
ity. 

The following details about coronary artery bypass graft 
25 surgery are largely derived from: Kirklin JW f Barratt-Boyes BG. 
Cardiac Surgery - Morphology, Diagnostic Criteria, Natural History, 
Techniques, Results, and Indications. New York: Churchill Living- 
stone, second edition, 1993 (for bypass surgery pp. 73-112; 143- 
147; 175-177; 299-316). 

30 

If drug treatment of the consequences of coronary artery 
disease fails, the patient is treated by percutaneous transluminal 
coronary angioplasty (PTCA) or by coronary artery bypass graft 
( CABG ) surgery. In 1991, in the US about 300,000 CABG operations 
35 were performed and approximately the same number of PTCAs. In 1991, 
the CABG operation and the subsequent 7-10 days in hospital was in 
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the US associated with an average cost of $40,000. Thus, the cur- 
rent US expenditure for CABG exceeds 12 billion dollar. 



The current CABG operation consists of the following sequence 
5 of steps: opening the chest by median sternotomy and spreading left 
and right rib cage; mobilization from chest wall of the left (and 
right) internal mammary artery (arterial grafts) and removal of 
greater saphenous vein from the leg (venous grafts); opening peri- 
card; arterial and venous cannulation for cardiopulmonary bypass 

10 (CPB); start extracorporeal circulation by the heart-lung machine 
which oxygenates the blood and produces the perfusion pressure for 
the body; cooling body to 32°C; aortic cross clamping and cold (4 
degrees) cardioplegic perfusion of the coronary arteries via the 
aortic root to arrest and cool the heart; attachment of on average 

15 one arterial and three venous distal anastomoses (the distal bypass 
graft end is connected to the coronary artery beyond the obstruc- 
tion; the proximal bypass graft end is attached to the root of the 
aorta, unless the graft is the internal mammary artery, in that 
case the bypass blood supply originates from the subclavian artery, 

20 a major branch of the aortic arch) ; removal of aortic crossclamp 
and warming up; attachments of proximal anastomoses to ascending 
aorta after applying side biting clamp on root aorta and punching 
holes in the aortic wall; weaning from CPB and decannulation; 
closing chest. The operation usually requires about 3.5 hours. 

25 Cardiac arrest lasts about 1 hour. The bypass surgery itself 
requires some 80 minutes. Thus, conditioning the patient for CABG 
surgery and the subsequent deconditioning takes most of the operat- 
ing room time. 

30 To appreciate the fundamental gains that potentially result 

from the present invention, the problems involved in these prepara- 
tory steps are listed in some detail. 

Cardiopulmonary bypass (CPB) for cardiac surgery is concept- 
35 ually simple, and equipment is now available to accomplish it with 
ease. To enable arresting the heart, all the patient's systemic 
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blood, which normally returns to the right atrium, is diverted into 
a device (oxygenator) in which oxygen is supplied to the blood and 
carbondioxide is removed. The newly arterialized blood is pumped 
from the oxygenator into the patient* s aorta at adequate blood 
5 pressure for further distribution to the body. The extracorporeal 
circuit includes, in addition to the oxygenator and several pumps, 
a reservoir, a temperature exchange system to lower or increase the 
temperature of the blood, several filters and pressure, flow and 
temperature sensors. 

10 

The problems of CPB result from the facts that blood is unac- 
customed to travelling through nonendothelially lined channels, to 
receiving gaseous and particulate emboli, and to experiencing non- 
physiologic shear stresses. In particular, exposure of blood to 
15 foreign surfaces results in the activation of virtually all the 
humoral and cellular components of the inflammatory response, and 
some of the more slowly reacting specific immune responses are 
initially activated as well. 

20 Details of this generalized response to extra-corporeal cir- 

culation include: neutrophil activation; platelet activation with a 
40% drop in circulating platelets; depression of platelet function 
with sometimes strong bleeding tendency; complement activation; 
kallikein activation and bradykinin release; activation of the 

25 fibrinolytic cascade which may contribute to postoperative bleed- 
ing; enhanced production of interleukin-1 and Tissue Necrosis Fac- 
tor. 

Additional complications of CPB include: loss of red blood 
30 cells (haemolysis) due to shear stress damage in the heart-lung 
machine; defective coagulation postoperatively because protamine 
only partly restores the coagulation cascade which was inhibited by 
heparin prior to CPB; in rare cases, (fatal) anaphylactic shock 
(low blood pressure) due to complement activation by protamine. The 
35 metabolic response to the stress of CPB is reflected in the large 
rise of plasma catecholamines during CPB. 
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Finally, establishing CPB adds 15 minutes to the operating 
room time and it adds to the cost of CABG by the required assist- 
ance of a perfusionist during the entire procedure and by the 
expenses for both the standard equipment (roller pumps , etc) and a 
5 number of expensive disposables (oxygenator, etc). 

With respect to cross-clamping the aorta the following 
remarks may be made. Prior to putting the patient on full CPB in 
order to start cold cardioplegic cardiac arrest, the aorta is 

10 clamped proximal to the insertion site of the arterial CPB cannula. 
The application of the aortic cross-clamp carries more than the 
usual hazard in patients undergoing CABG, because of the frequent 
presence of arteriosclerosis in the ascending aorta in these 
patients and the tendency of the cross-clamping to produce arterio- 

15 sclerotic emboli to the brain (about 0.5% embolic complications). 

In spite of deep cooling of the heart, cold cardioplegic 
cardiac arrest, in combination with the CPB induced generalized 
inflammatory response, may result in temporary dysfunction of car- 

20 diac muscle cells (stunned or hibernating myocardium) or in perma- 
nent damage of the often already jeopardized heart. Insufficient 
capacity of the heart to pump an adequate amount of blood into the 
aorta is a life threatening hypotensive condition which requires 
intensive treatment. It is the major cause of hospital mortality 

25 (1-3%, depending on risk factors). 

The principal disadvantage of open chest surgery is to the 
patient: almost 1% chance of sternal infection which severely pro- 
longs hospital stay and adds $42,000 to the hospital bill, and 
30 post-operative pain due to spreading of the ribcage. The post-oper- 
ative pain may lead to inadequate coughing and results in a 10-20% 
risk of pneumonia which also extends the hospital stay and requires 
expensive antibiotic treatment. 

35 m addition to cardiopulmonary bypass, the patient undergoing 

cardiac surgery experiences all the stress responses characteristic 
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of major operations and trauma. 

Most patients survive cardiopulmonary bypass and the oper- 
ation, and convalesce in a reasonably normal manner. Even then, 
5 however, -every patient retains for a time a few demonstrable stig- 
mata from the procedure, some have major morbidity, and a few die 
from their response to CPB and the surgery. The prevalence of these 
unfavourable outcome events in a group of patients is in part 
determined by the prevalence of identified risk factors, but the 
10 determinants of their occurrence and severity in an individual 
patient have not been completely described. 

After isolated CABG operations, in a heterogeneous group of 
patients the one month death rate is 3.5%. The major cause of death 

15 and complications are failure of the heart muscle to recover ade- 
quately from the cold arrest in combination with the generalized 
inflammation response and deregulation of haemostasis. The low 
output state is associated with intensive drug treatment, aortic 
balloon pumping and implantation of temporary cardiac assist 

20 devices, all measures which add to the cost of the procedure. 

During the first 6-8 weeks of convalescence from the CABG 
operation, patients commonly have a poor appetite, insomnia, emo- 
tional depression; visual, memory, or intellectual deficits; loss 

25 of sexual ability; lack of desire to return to work; and other 
potentially disabling manifestations of the postoperative states 
Studies have documented the transient nature of most of these phe- 
nomena. Except in unusual circumstances, patients who were active 
and gainfully employed before surgery can return to full activity 

30 and employment within 2 to 3 months after surgery. 

In the past two decades, to overcome the problems mentioned 
above, some cardiac surgeons have operated directly on the beating 
heart without CPB and without cardiac arrest, as is disclosed in: 
35 _ Kluge TH, Kerth WJ, Gerbode F. Aorto-coronary bypass in 

experimental animals without the use of extracorporeal circulation. 
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Scand J Thoracic Cardiovasc Surg 1972:6:257-61; 

- Trapp WG, Bisarya R. Placement of coronary artery bypass 
graft without pump oxygenator. Ann Thorac Surg 1975;19:1-9; 

- Akins CW, Boucher CA, Pohost GM. Preservation of interven- 
5 tricular septal function in patients having coronary artery bypass 

grafts without cardiopulmonary bypass. Am Heart J 1984;107:304-9; 

- Benetti FJ. Direct coronary surgery with saphenous vein 
bypass without either cardiopulmonary bypass or cardiac arrest. 
Cardiovasc Surg 1985;26:217-20; 

10 - Buffolo E, Andrade JCS, Branco JNR, Aguiar LF, Ribeiro EE, 

Jatene AD. Myocardial revascularization without extracorporeal 
circulation: seven years experience in 593 cases. Eur J Cardio- 
thorac Surg 1990;4:504-8; 

- Pfister AJ, Zaki MS, Garcia JM et al * Coronary artery 
15 bypass without cardiopulmonary bypass. Ann Thorac Surg 1992; 5- 

4:1085-92. 

The distal anastomosis is made after immobilization of the 
target artery with traction sutures. However, the coronary artery 

20 has to be clamped to do the arteriotomy and suture the anastomosis. 
As a result, this approach is limited to a subset of patients which 
are selected on the presence of occluded or almost occluded, well 
collateralized and easily accessible left anterior descending and 
right coronary arteries. This rare approach followed by the authors 

25 mentioned above is controversial because of poor results by some 
groups which is attributed to the trauma of immobilizing and clamp- 
ing the coronary artery. 

An image-directed robotic system for precise robotic surgery 
30 has recently been described by Glasmann et al. (US Patent 
5,086,401)* It involves the presurgical planning and shaping of a 
bone implant for hip joint replacement based on computer tomography 
data and the subsequent drilling during surgery of the properly 
dimensioned hole in the leg bone by means of a programmed robotic 
35 drill. The robotic surgery is limited to programmed drilling on an 
immobilized bone. It does not involve surgery on a moving object 
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and it does not include the combination of automated and superim- 
posed surgical movements of the robotic instruments guided by 
stereoscopic video input to the surgeon of the surgical target 
area. 

5 

Summary of the invention 

A primary object of the present invention is to provide a 
system for close inspection in real time of moving parts, e.g., 
1 0 moving elements of a machine . 

A further object of the present invention is to provide a 
medical system for close inspection in real time of a moving object 
within a chest cavity without the need to open the chest. 

15 

A further object of the present invention is to provide a 
system for precise manipulation of the moving object within the 
chest cavity, e.g. robotic precise surgery on the moving object, 
without the need to open the chest. 

20 

Still, a further object of the present invention is to pro- 
vide a medical system for closed chest robotic coronary artery 
bypass grafting on a beating heart. 

25 To that purpose, the present invention provides an inspection 

system comprising: 

- at least a first and a second camera to be mounted in a 
fixed position relative to each other and to direct their respect- 
ive optical ranges to a moving target; 

3Q - an image processor receiving output signals from both the 

first and second cameras, the output signals containing video 
images of the moving target, the image processor being programmed 
to 

* detect reference locations of beacons on the moving target 
35 area in a reference video image, as well as locations of the bea- 
cons in each subsequent video image; 
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* compute a set of distances between the locations of the 
beacons in each subsequent video image. and the corresponding refer- 
ence locations, the set of distances being used to translate and 
rotate the locations of the beacons in each subsequent video image 
5 in order to match them with the reference locations and to obtain a 
substantially arrested image of the moving target area (virtual 
target image arrest ) ; 

- display means connected to the image processor to receive 
and display the arrested image of the moving target area. 

10 

The main advantage of such a system is that it provides means 
to inspect accurately a moving part, e.g.; of a machine without the 
need to stop the movement of that part. 

15 in another embodiment the invention provides an inspection 

and robotic tracking system comprising: 

- at least a first and a second camera to be mounted in a 
fixed position relative to each other and to direct their respect- 
ive optical ranges to a moving target; 

20 - an image processor receiving output signals from both the 

first and second cameras, the output signals containing video 
images of the moving target, the image processor being programmed 
to 

* detect reference locations of beacons on the moving target 
25 area in a reference video image, as well as locations of the bea- 
cons in each subsequent video image; 

* compute a set of distances between the locations of the 
beacons in each subsequent video image and the corresponding refer- 
ence locations, the set of distances being used to translate and 

30 rotate the locations of the beacons in each subsequent video image 
in order to match them with the reference locations and to obtain a 
substantially arrested image of the moving target area (virtual 
target image arrest); 

- display means connected to the image processor to receive 
35 and display the arrested image of the moving target area. 

- at least one robot arm to be mounted in a fixed position 
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relative to the first and second cameras; 

- at least one manipulation instrument to be connected to the 
robot arm and to a robotic computer system to control its actions; 

• a tracking control unit connected to the image processor 
5 and to the robot arm in order to supply the robot arm with control 
signals determined by output signals of the image processor; 

- at least one control robotic instrument to be manually 
operated and connected to the robotic computer system to supply the 
robotic computer system with control signals. 

10 

By applying these measures, one may, e.g., repair a moving 
part of a machine without interrupting that machine. There may be 
situations in which interrupting a machine will cost a lot of 
money, which may, therefore, now be saved. 

15 

In the inspection and robotic tracking system defined above 
the set of distances being used to translate and rotate the loca- 
tions of the beacons in the subsequent video images in order to 
match them with the reference locations and to obtain a substan- 

20 tially arrested image of the moving target area is converted by the 
image processor into control signals for the robot arm in such a 
way that, when no human operator operates the control robotic 
instrument, the manipulation instrument connected to the robot arm 
substantially tracks the moving target area, i.e. the position of 

25 the manipulation instrument relative to the moving target area 
remains substantially the same. 

in a preferred embodiment the present invention provides a 
medical system comprising: 
30 - at least a first and a second endoscopic video camera 

mounted on a first and a second mount, respectively, to be posi- 
tioned in a chest cavity in such a way that they both observe a 
predetermined moving target area within the chest cavity; 

• an image processor receiving output signals from both first 
35 and second video cameras, the output signals containing video 
images of the moving target, the image processor being programmed 
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to 

* detect reference locations of beacons on the moving target 
area in a reference video image, as well as locations of the bea- 
cons in each subsequent video image; 
5 * compute a set of distances between the locations of the 

beacons in each subsequent video image and the corresponding refer- 
ence locations, the set of distances being used to translate and 
rotate the locations of the beacons in each subsequent video image 
in order to match them with the reference locations and to obtain a 
10 substantially arrested image of the moving target area (virtual 
target image arrest ) ; 

- display means and being connected to the image processor to 
receive and display the arrested image of the moving target area. 

15 By applying these measures, a rapid close stereoscopic in- 

spection or diagnostics in real time of moving body parts within a 
chest cavity are possible without the need to arrest the moving 
body part in reality. These measures constitute important precondi- 
tions for remotely operating on a moving body part. 

20 

The beacons may be anatomic locations within the moving tar- 
get area. However, also artificial beacons may be installed on the 
moving target area. 

25 In a preferred embodiment of the invention, the medical sys- 

tem defined above further comprises a main camera mounted on an 
adjustable ball bearing to be inserted into the chest between 
neighbouring ribs and to supply video image information of any area 
within the chest to at least one video monitor. These measures 

30 provide the operating team with a real time unmodified overview of 
the target area and its surroundings to allow the team to see what 
body parts are moving or standing still in reality. 

The image processor may be connected to a voice activator 
35 receiving voice signals from a microphone, the voice activator, 
during operation, generating zooming-in or zooming-out commands for 
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zooming-in or zooming-out the video image of the first and second 
cameras. By the application of these measures the primary surgeon 
does not need his hands for zooming-in on a desired target area. 



5 In a further preferred embodiment the medical system further 

comprises : 

- at least one robot arm to be mounted in a fixed position 
relative to the first and second cameras; 

- at least one robotic surgical instrument to be connected to 
10 the robot arm and to a robotic computer system to control its 

actions; 

- a tracking control unit connected to the image processor 
and to the robot arm in order to supply the robot arm with control 
signals determined by output signals of the image processor; 

15 - at least one control robotic instrument to be manually 

operated and connected to the robotic computer system to supply the 
robotic computer system with control signals. 

In such a preferred embodiment the set of distances being 
20 used to translate and rotate the locations of the beacons in the 
subsequent video images in order to match them with the reference 
locations and to obtain a substantially arrested image of the mov- 
ing target area is converted by the image processor into control 
signals for the robot arm in such a way that, when no human oper- 
25 ator operates the control robotic instrument, the surgical in- 
strument connected to the robot arm substantially tracks the moving 
target area, i.e. the position of the surgical instrument relative 
to the moving target area remains substantially the same. 

30 Preferably, the medical system also comprises ultrasound 

means to locate and evaluate a coronary artery on a beating heart, 
and at least two electromotor driven, rotating ultrasound trans- 
ducers (20-60 MHz), the ultrasound means being provided with suc- 
tion means to fix the ultrasound means to the beating heart. 

35 

The ultrasound means may have a hole to receive a robotic 
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stapler/suturing device with one end of a bypass graft. 



The robotic stapler/suturing device may comprise an adjust* 
able expansion ring and a thick walled cylinder provided with an 
5 aperture to receive the bypass graft to bypass an obstructed part . 
of the coronary artery , wherein an end part of the bypass graft is 
to be expanded in diameter by the expansion ring and folded back 
around the thick walled cylinder by the expansion ring, and wherein 
the thick walled cylinder , during operation, accommodates bonding 
10 means to bond the end part of the bypass graft to the coronary 
artery (distal anastomosis) or to an aorta (proximal anastomosis). 

Preferably such a medical system comprises hole making means 
to be inserted in the bypass graft through a temporary side branch 
15 after the bypass graft is bonded to the coronary artery. 

By way of example the hole making means comprises a circular 
cutter within a cutter housing, which allows substantially resis- 
tance free rotation of the circular cutter within the bypass 
20 artery. 

Moreover, the medical .system may comprise other hole making 
means to make a hole in an aortic wall, the other hole making means 
comprising a first circular rotary cutter driven by a flexible 

25 cable and a contra cutter with an exceedingly sharp screw front, 
the contra cutter being housed within the first circular cutter, 
being driven by another flexible cable housed within the first 
flexible cable, and to be screwed into and through the aortic wall 
to catch a circular piece of aortic wall cut out by the first cir- 

30 cular cutter. 

Summarizing, among other things, the invention discloses an 
entirely novel method of performing coronary artery bypass graft 
(CABG) surgery by means of a minimally invasive (thoracoscopic) 
35 approach (MI CABG) without the need for cardiopulmonary bypass 
(CPB), cardiac arrest and opening and spreading the ribcage. All 
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procedures are performed with standard thoracoscopic techniques, 
but for clearing of the distal anastomosis sites if necessary and 
positioning an ultrasound guided anastomosis device (USGAD) con- 
taining a robotic stapler /suturing device and effecting the anasto- 
5 mosis bonding. 

The method employs robotic tracking of cardiac surface motion 
to enable attaching the graft (donor vessel) onto the moving reci- 
pient coronary artery distal to its obstruction. The end-to-side 
10 anastomoses are made with a robotic, ultrasound guided sutur- 
ing/stapling device using an approach first described (and applied) 
on static arteries by Tulleken using current manual suturing tech- 
niques in: 

- Tulleken CAF, Dieren Avan, Verdaasdonk RM: A new type of 
15 end to side anastomosis between arteries using the Neodynium-YAG 

laser with a sapphire tip (abstr). Lasers Med Sci 1998;3:335; 

- Tulleken CAF, Verdaasdonk RM, Dieren Avan: End-to-side 
anastomosis without occlusion of the recipient artery, using the 
Neodynium YAG-laser with a sapphire tip (abstr). Lasers Med Sci 

20 1991;6:88-9; 

- Tulleken CAF, Van Dieren A, Verdaasdonk RM, Berendsen W. 
End-to-side anastomosis of small vessels using an Nd : YAG laser with 
a hemispherical contact probe (technical note). J Neurosurgery 
1992;76:546-9; 

25 - Tulleken CAF, Verdaasdonk RM: Use of the excimer laser in 

the end-to-side anastomosis between brain vessels. In: Anderson RR 
(ed), Laser Surgery: Advanced Characterization, Therapeutics and 
Systems III. Bellingham: SPIE vol. 1643, 1992, pp. 205-12; 

- Tulleken CAF, Verdaasdonk RM: Use of excimer laser in high 
30 flow bypass surgery of the brain (abstr). Laser Medizin 1992;8:138; 

- Tulleken CAF, Verdaasdonk RM, Berendsen W, Mali WPTM: Use 
of the excimer laser in high flow bypass surgery of the brain. J 
Neurosurg 1993;78:477-80. 

35 The Tulleken approach obviates the need to interrupt the 

blood flow in the recipient artery. By sizing all instrumentation 
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to the intercostal rib space , the procedure can be performed 
thoracoscopically . The present surgical approach is modified from 
the novel end-to-side anastomosis technique described by Tulleken 
et al. and relates to end-to-side anastomoses but is, in principle, 
5 applicable to side-to-side anastomoses (jump grafts) as well. 

Benefits of the robotic Minimally Invasive Coronary Artery 
Bypass Graft (MICABG) system are: 

I . no extracorporal circulation; 

10 2. no cold cardioplegic cardiac arrest; 

3. no need for perfusionist; 

4. no need for expensive disposables associated with CPB; 

5. no need to cross-clamp or side-clamp the aorta; 

6. shorter grafts on the left side because the proximal anastomosis 
15 is on the aorta descendens rather than on the aorta ascendens; 

7. reduced mortality of CABG surgery; 

8. reduced morbidity; 

9. reduced need for intensive treatment of haemostatic disorders or 
cardiac failure postoperatively; 

20 10. reduced hospital stay (from 7-10 days to 4 days); 

I I . reduced reconvalescence period and earlier return to normal 
activities; 

12. reduced overall costs of the operation and hospitalization. 

13. no large mid line thorax wound from neck to abdomen and 
25 subsequently, no large scar. 

Brief description of the drawings 

The above set forth and other features of the invention are 
30 made more apparent in the ensuing Detailed Description of the 
Invention when read in conjunction with the attached Drawings, 
wherein : 

Figure 1 is a block diagram of the presently preferred em- 
35 bodiment of endoscopic stereoscopic (3-D) video imaging of a surgi- 
cal target, together with the possibility to view an entire oper- 
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ation field on a video monitor or to view an operating room. 

Figure 2A schematically depicts a beating heart and immobile 
other structures in a chest cavity. 

5 

Figure 2B schematically depicts effects of virtual target 
image arrest with in the image a virtually arrested heart and 
•beating' structures in a 'beating 1 chest cavity. 

10 Figure 3 schematically depicts the surgical target area on 

the beating heart with traction sutures to reduce coronary- artery 
motions and motions of beacons in the vicinity of the surgical 
target area. 

15 Figure 4 is a block diagram showing arbitrary positions of 

beacons in a reference video image and in subsequent images; by 
matching the position of the beacons in the latter images to the 
beacons in the reference image, virtual target image arrest is 
obtained. 

20 

Figure 5 is a block diagram of virtual target image arrest 
combining figs. 1, 3 and 4, illustrating EKG triggering for the 
reference image. 

25 Figure 6 is a schematic diagram of operator controlled 

(scaled) motions of surgical instruments which are superimposed on 
automated target tracking movements of robotic arms with surgical 
instruments. 



30 



Figure 7 illustrates attachment of an expandable ring to an 
end of a bypass graft. 

Figure 8 is a schematic diagram of a robotic stapler/suturing 
device around the end of the graft. 



35 



Figure 9 is a schematic cross 



-section of the end of the 
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robotic stapler/suturing device and the everted end of a graft. 

Figure 10A is a schematic drawing (transparent top view) of 
an ultrasound guidance device with two electromotor driven rotating 
5 ultrasound transducers adjacent to a central hole which can accom- 
modate a robotic stapler/suturing device with an end of the bypass 
graft. 

Figure 10B provides a longitudinal section of the ultrasound 
10 guidance device of Figure 10A. 

Figure 10C provides a cross-section of the ultrasound guid- 
ance device at the level indicated by arrow C in figure 10B. 

15 Figure 10D provides a cross-section of the ultrasound guid- 

ance device at the level indicated by arrow D in figure 10B. 

Figure 10E provides a cross-section of the ultrasound guid- 
ance device at the level indicated by arrow E in figure 10B. 

20 

Figure 11 is a schematic longitudinal section of the ultra- 
sound guided anastomosis device (USGAD) with the bypass graft (with 
temporary side branch), its end folded back over the robotic sta- 
pler/suturing device which fits into the ultrasound guidance 
25 device. 

Figure 12A is a magnification of the robotic stapler end from 
fig. 11 after ejection of staples. 

30 Figure 12B gives a schematic top view of circumferential 

staple bonding. 

Figure 12C illustrates a preformed staple which has 
penetrated the graft wall at both ends and which remains within the 
35 wall of a recipient artery. 
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Figure 12D illustrates a preformed staple which has 
penetrated the graft wall at one end and folds back on itself. 



Figure 13 shows a schematic drawing of a circular cutter 
5 introduced through the temporary side branch of the bypass graft to 
establish continuity between graft lumen and recipient coronary 
artery lumen by punching a hole in the coronary artery wall. 

Figure 14 shows, similar to Figure 13, a schematic drawing of 
10 a circular (or ellipsoid) spark erosion electrode to ablate a ring 
of coronary artery wall tissue to punch a hole. 

Figure 15 shows a schematic drawing of a cutter and contra- 
cutter with sharp screw-end of an aortic wall punch. 

15 

Detailed description of the invention 

Endoscopic stereoscopic (3-D) video imaging viewed through 
operating spectacles is shown in Figure 1 . The preferred thoraco- 
20 scopic surgical technique uses two intrathoracic CCD cameras 1, 2. 
To allow the surgeon to see depth, stereoscopic video imaging is 
implemented to the separate eyes in operating spectacles 111. 

The two directional endoscopic CCD cameras 1, 2 are posi- 
25 tioned on ball bearings 3, 4 at the end of mounts 5, 6, respective- 
ly. The mounts 5, 6 are inserted through the chest between two 
neighbouring ribs, one of which 110 is shown in Figure 1. The 
mounts 5, 6 are mounted on a cross bar 7 at variable distance and 
right angles. The bearings 3, 4 are externally controlled like 
30 eyes. First, ball bearing 3 is adjusted by the surgeon to direct 
CCD camera 1 to get the target segment 22 of the coronary artery 23 
in the middle of its video image. Next, ball bearing 4 is adjusted 
to direct CCD camera 2 to get the target segment 22 of the coronary 
artery 23 in the middle of its video image. The angles a, £ are 
35 defined by the normal to the cross bar 7 and the normals to the 
input planes of CCD cameras 1, 2, respectively. The angle y is 
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defined by the plane through mounts 5, 6 and the plane through the 
normals to the input planes of CCD cameras 1, 2. The angles a, p, y 
are monitored and fed to a digital image processor 8. The adjust- 
able distance between the CCD camera mounts 5, 6 on the cross bar 7 
5 is monitored and fed to the digital image processor 8. From angles 
a, p, y 2 nd the distance between CCD cameras 1 and 2 (mounts 5, 6), 
the distance calibration in the stereoscopic video image is com- 
puted by image processor 8. This embodiment will optimize stereo- 
scopic vision of the target over a sufficient space angle and com- 
10 ply with anatomical constraints on making access holes in the chest 
wall in between ribs. Illumination of an object within the chest is 
provided separately in standard thoracoscopic fashion by means of 
light sources 9 and 10. 

15 two synchronized video systems 11 simultaneously produce the 

image of the first CCD camera 1 and the image of the second CCD 
camera 2. Both images are fed to the digital image processor 8 
which creates simultaneously the digital output to "operating spec- 
tacles" 111 (Figure 1 ) . In these "spectacles" a usual optical lens 

20 system (magnification about 3x) has been replaced by a "left" and a 
"right" video image, respectively. Video image display is by color 
liquid crystal display and optical means or by other means which 
provide adequate resolution of a color image in about 2-4 cm 2 . 

25 a primary surgeon 101 has a microphone 12 which allows voice 

activation of the digital zoom capability of the images from the 
CCD cameras 1, 2 by a voice control unit 13 through the image pro- 
cessor 8. The digital zoom of the images from the CCD cameras 1, 2 
does not need to be centred at the centre of the image. By voice 

30 activation, the centre of the zoom field is called to be displayed 
as cursor in the center of the image and, next, is moved by voice 
command to a point whose coordinates are read at the edges of the 
video image. Zooming-in will scale the original coordinates. These 
coordinates are derived from the angles a, p and y involved in the 

35 CCD camera 1, 2 positions as well as from the distance between CCD 
cameras 1 and 2. Of course, the zooming may be provided by other 
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means than by voice activation, e.g. by a foot switch. 

By providing other members 102, 103, 104 of the operating 
team and any visitors 105, 106 with such operating spectacles 111, 
5 the endoscopic operating procedure can be closely followed by all 
in 3-D. 

The left and right (zoom) images may be recorded on two syn- 
chronized video recorders 14, 15 coupled to the image processor 8, 
10 which are also preferably activated by voice command through voice 
control unit 13. These recorders 14, 15 may monitor an audio chan- 
nel as well . 

In addition to the stereoscopic video image in the operating 
15 spectacles system, the entire surgical area of interest is moni- 
tored by a standard thoracoscope 107 which has a CCD camera 19 
mounted on a ball bearing 18 to allow vision in all directions. 
General illumination of the object region is provided by a main 
light source 108, which supplies light in the standard thoraco- 
20 scopic fashion. The output of CCD camera 19 is fed to the video 
system 17, which has e.g. 4 monitors connected: a first monitor 16 
at the head of the operating table (not shown), a second monitor 
16a at the right side of the operating table, a third monitor 16b 
at the left side of the operating table, and a fourth monitor 16c 
25 for the cardio-anaesthesiologist . In this way all members 101-104 
of the surgical team have both an overview of the entire surgical 
field within the thorax and a magnified stereoscopic view of the 
grafting area. 

30 The major advantage of incorporating the left and right video 

images in operating spectacles 1 1 1 is that a cardiac surgeon is 
used to perform CABG surgery with operating spectacles which mag- 
nify the target region about 3 times, but which leave peripheral 
vision undisturbed. As a result, in the new approach described here 

35 the surgeon can concentrate on the target, use zooming in if neces- 
sary, and at the same time, keep visual contact with a monitor 16 
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displaying, for instance, the general view of the heart and chest 
cavity, with the monitors displaying the EKG and haemodynamic para- 
meters of the patient, with his hands and the instruments outside 
the body, and with the other people in the operating room. 

5 

Figures 2 to 5 relate to virtual cardiac target image arrest 
which will now be explained. The surface of e.g. a beating heart 20 
(Fig. 2A) shows (regular) motion with respect to the chest cavity 
21 and other objects which are stationary with respect to the oper- 

10 ating table (not shown). Hence, the potential site of the distal 
anastomosis of the bypass graft, the target area 22, moves with 
respect to the chest wall, the thoracoscope 107, CCD cameras 1, 2 
and thoracoscopic instruments. It is preferred to partly immobilize 
the coronary artery segment (Fig. 3) which is to be grafted 23. To 

15 that purpose, by current endoscopic techniques several traction 
sutures 25 are placed around the target area 22, which is distal to 
a (partially) obstructed part of a coronary artery 23. 

In the vicinity of the target area 22, beacons 24 are ident- 
20 ified. The beacons 24 may be clearly identifiable anatomic struc- 
tures or clips placed on the surface of the heart 20 or e.g. tiny 
LED 1 s which are temporarily attached to the surface of the heart 
20. 

25 The patient is under beta-blockade to reduce heart rate and 

reduce the electrical irritability of the cardiac muscle to mechan- 
ical stimulation. If in spite of the anaesthesia, the heart rate or 
rhythm fluctuates considerably, a temporary right heart pacing 
electrode may be attached to the right auricle to pace the heart 

30 slightly above its own highest rate. 

At end-diastole one video image (left and right) is frozen, 
preferably, by voice command. End-diastolic freezing of the video 
image is accomplished by triggering on the QRS complex of the elec- 
35 trocardiogram (EKG) with an appropriate delay. Beacons 24 are 
defined interactively (mouse or joy stick controlled cursor in 
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video image) on the surface of the heart near the edge of the tar- 
get area (Fig. 3). Their 3-D coordinates are computed relative to 
the coordinate system of the two video cameras 1 , 2 . This 
coordinate system is defined by the plane through cross bar 7 and 
5 the mounts 5, 6, with the origin in the middle of cross bar 7. In 
each following stereoscopic image the beacons are recognized by 
fast computer image analysis algorithm (Fig. 4). 

Figure 4 shows the procedure how the image processor 8 pro- 
10 cesses the stereoscopic video images received from CCD cameras 1, 
2. First, an image is chosen as reference image 115 with beacons 
24, as explained below. In all subsequent images, e.g. image 116, 
the beacons 24' are displaced relative to the positions of the 
beacons 24 shown in video image 115. Since the actual displacement 
15 of the heart is three dimensional, the displacements of the beacons 
24' in video image 116 relative to their original positions in 
video image 115 are also three dimensional. 

Video image 1 1 7 in Figure 4 shows a coinciding stereoscopic 
20 image of video images 115, 116. The arrows P denote the 3D dis- 
placements of beacons 24 1 in video image 116 to the original bea- 
cons 24 in video image 115 . Video image 116 shows beacons 24' 
matching beacons 24 when beacons 24" are virtually shifted back 
along arrows P (and rotated) to their original positions in video 
25 image 115. Why beacons 24' are virtually shifted back to their 
original positions will be explained below. 

Figure 5 shows a flow chart of virtual target arrest by image 
processing by image processor 8. In step 120, video images of the 
30 moving surgical target 22 on the beating heart 20 are received by 
video processor 8. In step 121, an end-diastolic image, for 
-instance video image 115 (Figure 4), is frozen using EKG triggering 
122 input to the image processor 8 (not shown). After unfreezing 
the video display, in the original video image 115 the beacons 24 
35 are identified. In step 124, in each subsequent video image, e.g. 
image 116 (Figure 4), the positions of the beacons 24' are recog- 
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nized and determined. Next, the 3-D distance between each of the 
beacons 24 in video image 115 and the corresponding beacons 24* in 
video image 116 are computed in step 125. Then, in step 126 the 
video image 116 is translated and rotated in such a way by appro- 
5 priate computations that video images 115 and 116 match each other, 
as shown in video image 118 in Figure 4. This process is repeated 
with each subsequent video image resulting in virtual image arrest 
(cardiac target arrest) in step 127. As a consequence, non-cardiac 
structures will appear to be beating (Figure 2B), when the target 
10 arrested video image 127 is transmitted 128 to the spectacles 111 
of the surgeon 101 (Figure 1). 

An absolute condition for virtual target arrest is that the 
target area and the nearby beacons never disappear from the orig- 
15 inal video image during the cardiac cycle and the respiratory 
cycle. 

If robotic tracking of the target (vide infra) is not fast 
enough due to a relatively long interval between video-images (40 
20 ms), the standard frame rate (25 Hz) is increased, or detection and 
tracking of beacons by other means is employed. 

Figure 6 shows how the system shown in Figures 1-5 may be 
used during an operation of a body part within the chest, for 

25 instance, a coronary artery 23. The Figure shows the two mounts 5, 
6 for the CCD cameras 1 and 2, respectively, which transmit their 
video signals to the image processor 8. For the sake of clarity, 
the light sources 9, 10 and the synchronized video systems 11 as 
depicted in Figure 1 have been omitted in Figure 6. By using the 

30 method of Figure 5, image processor 8 generates a video image of 
the virtually arrested object of interest, e.g. the target segment 
22 of coronary artery 23. The arrested video image is outputted to 
the spectacles 111 of at least the surgeon 101. Then, the computa- 
tional algorithm to obtain the arrested target image is employed in 

35 real time to translate and rotate robot arms 32, 33 with robotic 
surgical instruments 34, 35, the robot arms 32, 33 being inserted 
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between two neighbouring ribs like 110 in the same way as the two 
mounts 5, 6 of CCD cameras 1, 2. The robot r.rms 32, 33 with their 
robotical surgical instruments 34 , 35 receive tracking signals from 
tracking control 31 in order to translate and rotate them with 
5 respect to the coordinate system in precisely the opposite way as 
does the video image 116 (Figure 4) to match video image 115. As a 
result, the robotic tracking arms 32, 33 with their robotic surgi- 
cal instruments 34, 35 track the target 22 in real time. In other 
words, if the surgeon 101 does not interfere, the (small) distance 
10 between the surgical instruments 34, 35 and the target 23 remains 
constant, whatever the movements of the target area 23. The track- 
ing of each of the two arms 32, 33 may preferably be switched 
on/off by voice control unit 13. 

15 To insert the robot arms 32, 33 with surgical instruments 34, 

35 through appropriate holes between neighbouring ribs, they will 
either have to be inserted before fixing them to crossbar 7 or will 
have to be connected to crossbar 7 by suitable fixing means (not 
shown) allowing the freedom of insertion through the holes without 

20 being limited by the crossbar 7, which is afterwards connected to 
the operating table in a stable fashion (not shown). To exchange a 
surgical instrument 34, 35/ the instrument is disconnected either 
within or outside the chest from robotic arm 32, 33 and exchanged. 

25 Preferably, during operating a patient the patient is laid on 

his back or on one of his sides and one lung is collapsed in order 
to create suitable operating space for CCD cameras 1, 2, 19, the 
robot arms 32, 33 and the instruments 34, 35. 

3q The surgeon 101 manually handles control robotic instruments 

36a, 36b (with left hand and right hand, respectively) (e.g. tweez- 
ers) which control a robotic computer system 37, which in turn 
steers by means of independent controls 38, 38' the output surgical 
instruments 34, 35 (robotic telesurgery) which are mounted on the 

35 tracking robot arms 32, 33. Any movement of the surgeon 101 with 
the control robotic instruments 36a, 36b is translated - with or 



BNSDOCID:<WO 9501757A1 I > 



WO 95/01757 




PCT/NL94/00156 



without voice command controlled scaling down of movements - to the 
robotic surgical instruments 34 , 35. The combined motion of robotic 
surgical instrument 34 resulting from the control signals generated 
by tracking control 31 and the control robotic instrument 36a 
5 results in operation on the moving target 22 by robotic surgical 
instrument 34. The same applies to surgical instrument 35. The 
surgeon 101, in contrast, experiences the procedure in his operat- 
ing spectacles 111 as operating on the arrested target, whereas one 
look over the rim of the operating spectacles 111 at the video 
10 monitor 16 will tell him that he is working on the moving target 
22. 

Now, a circular or ellipsoid robotic stapler/suturing device 
(ultrasound guided anastomosis device U5GAD ) will be described, 
15 referring to Figures 7-12. An expandable ring 41 (like key ring) is 
attached to the end of a bypass graft 40 by a running suture 42, 
and expanded (Figure 7). The end of the graft 40 now looks like a 
trumpet. The opening of this "trumpet" may be circular, however, 
also an ellipsoid form is possible and even preferred. 

20 

The graft 40 is then inserted into a hollow robotic 
stapler /suturing device 44 with staple/suturing control cable 131 
by squeezing the vessel gently in between an opening slit 45 in the 
stapler/suturing device 44 (Figure 8). Once the graft is inside the 

25 robotic stapler/suturing device 44, the expanded ring 41 is now 
retracted over the stapler device 44 while the latter is gently 
advanced until the ring 41 passes a ridge 47. With a suture 49 
proximal to the ridge 47, the end 39 of graft 40 is fixed to the 
stapler /suturing device 44 and the tension of the suture closes 

30 slit 45. The slit is designed to be opened after establishing the 
anastomosis bonding as exit for the bypass graft. Next, the running 
suture 42 around ring 41 is removed and the ring is withdrawn, hs a 
result, the everted graft wall 46 now fits the outside of the sta- 
pler /suturi ng device 44, as illustrated in Figure 9 which shows a 

35 cross section of the stapler device 44 and the everted graft 46. 
Staples 48 are present in the end part of the hollow cylindrical 
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stapler device 44 to be ejected from its end part, as will be 
explained below. The robotic stapler/suturing device may also and 
preferably end in a bevelled, ellipsoid configuration (not shown) 
to allow creating an anastomosis corresponding more to current CABG 
practice, which avoids kinking of the graft and which maximizes the 
area of the anastomosis opening. 



A high frequency (20-60 MHz) ultrasound analysis of the sur- 
face of the heart at the designated target area 22 to find the 

10 optimal anastomosis site on the coronary artery to be operated is 
performed as follows (Figure 10 A-E and Figure 11). High frequency 
(30 MHz) ultrasound analysis of the surface of the. heart and the 
underlying coronary artery is performed with methods similar to 
intravascular ultrasound imaging described in e.g. Borst C, Savalle 

15 LH, Smits PC, Post MJ, Gussenhoven WJ, Bom N . , "Imaging of post- 
mortem coronary arteries by 30 MHz intravascular ultrasound", Int J 
Cardiac Imag 1991;6:239-46. 

An ultrasound diagnostic device 150 is made of light weight 
20 materials. It consists of a rectangular block less than about 25 x 
12 x 5 mm) at the end of a flexible tube 151, which fits through a 
15 mm I.D. trocart (not shown). In the middle, an opening 152 can 
accommodate the stapler/suturing device as will be explained below 
(Figure 10 A, B and E). There are various sizes holes in different 
25 ultrasound devices to accommodate grafts on different diameter 
coronary arteries. At e.g. eight positions, suction openings 153 
connected through suction channel 154 to the lumen 155 of tube 151 
allows temporary attachment by subatmospheric pressure of the 
ultrasound device 150 to the moving surface of the heart, on top of 
30 coronary artery 23 underneath. 

The ultrasound device (Figure 10) contains proximally one 
ultrasound unit consisting of an electromotor 156 (diameter less 
than 3-4 mm, length less than about 8 mm) which is powered by the 
35 electrical cable 157. The electromotor 156 runs at about 4000 RPM, 
but through 4:1 gears 158, the speed is brought down to 1000 RPM. 
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In this way a plane circular 30 MHz (20-60 MHz) ultrasound trans- 
ducer 159 (diameter 1 mm) is rotated at 1000 RPM, the rotation 
speed of commercially available intravascular ultrasound devices. 
The ultrasound transducer is excited by high frequency pulses con- 
5 ducted by electrical cable 160 through sliding contacts 161 • The 
ultrasound system 162 is basically the same as the commercially 
available intravascular ultrasound system employed in Borst et al 
(1991) mentioned earlier. An electromotor driven rotary ultrasound 
element has been described by N. Bom and C.T. Lancee in a European 
10 Patent Application 423,895 with a miniature motor (1 mm diameter, 3 
mm long) to be incorporated in a coronary catheter. 

The ultrasound device (Figure 10)* contains a second ultra- 
sound unit distally at the end which consists identically of an 
15 electromotor 166, electric supply cable 167, 4:1 gears 168, an 
ultrasound transducer 169, ultrasound cable 170, and sliding con- 
tacts 171. The 4:1 gears 158 also allow positioning the ultrasound 
transducer 159, 169 closer to the ultrasound transparent TPX window 
163 (Figure 10B). 

20 

Preferably, the ultrasound transducer 169, 179 plane makes an 
angle of 15° with respect to the rotation shaft to avoid too large 
a reflection from the TPX window 163 through which the ultrasound 
beam is transmitted to the superficial layers of the heart 20 (Fig- 
25 ure 10 D) . The chamber 172 is filled with distilled water. 

The ultrasound device 150 is introduced in the chest cavity 
through an appropriate opening between two neighbouring ribs and 
handed over to a suitable surgical instrument 34 or 35 on one 

30 robotic tracking arm 32 or 33. The tracking control 37 is switched 
on and the ultrasound device 150 is lowered onto the prospective 
anastomosis site (the site has been determined previously on the 
basis of the coronary angiogram). Once the ultrasound device 150 
touches the surface of the heart, the eight suction channels 153 

35 are activated to help securing perfect tracking of the target area 
23 by the ultrasound device 150 (Figure 10C). 
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Once the ultrasound device 150 is properly attached to the 
surface of the beating heart, the proximal 159 and distal 169 
ultrasound transducers will display cross sections . of the recipient 
coronary artery 23 and its surroundings on both sides of the target 
5 anastomosis site 22. Both ultrasound transducers are connected to 
the ultrasound generating and analysing system 162. The ultrasound 
pulse frequency of 1000 Hz drives the transducers 159 and 169 each 
at 500 Hz by sending (and receiving), for example, the odd pulses 
to transducer 159 and the even pulses to transducer 169. The corre- 

10 sponding ultrasound cross-sections 181, 182, respectively, are 
displayed on videomonitor 180. If the ultrasound device 150 appears 
to be improperly positioned (not on top of the artery or if the 
place 22 shows too much plague in the recipient artery), its posi- 
tion is altered by stopping the suction and repositioning. The 

15 ultrasound device 150 allows optimizing the distal anastomosis, 
site, because the ultrasound cross-sections will reveal the actual 
plaque load 29 of the artery 23 (Figure 10D). Angiography, in con- 
trast, only provides a projection of the lumen. 



20 Once the ultrasound device 150 is attached to the surface of 

the heart over the coronary artery 23, it is left attached to the 
surface of the heart by means of the eight suction channels 153. 
Now the stapler/suturing device 44 having graft 40 retracted around 
its end part, as shown in Figure 9, is introduced into the chest 

25 cavity, again through an appropriate opening between two neighbour- 
ing ribs, handed to a surgical instrument 34 or 35 attached to a 
robotic tracking arm 32 or 33, lowered to the ultrasound device and 
positioned in the centre hole 152 of the ultrasound device. The 
ultrasound diagnostic device 150 (Figure 10) and the robotic sta- 

30 pier/suturing device 44 (Figures 8 and 9) combined form the ultra- 
sound guided anastomosis device, USGAD 190 (Figure 11). The ultra- 
sound cross-section of the artery 23 (during light pressure by the 
catheter) allows determination of the thickness 29 of the arterial 
wall of the artery 23 (Figure 10 D). 

35 

The proper stapler /suturing device 44 has been chosen as to 
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its diameter and its stapling depth. Note that the device 44 has 
been attached to the end of the graft outside the chest. This is 
also possible with the internal mammary artery (arterial bypass 
graft) after mobilization, because this artery is long enough to 
5 bring the distal end outside the chest. Besides, staples 48 have 
been inserted into the end part of the stapler device 44 outside 
the chest. After inserting the staples 48, staple ejecting means 
130 to be operated by press gas through the hollow stapler device 
44 are laid upon the staples 48, controlled from outside the chest 
10 by stapler /suturing control cable 131. 

The stapler/suturing device 44 is advanced until the ridge 47 
is blocked by the ultrasound device 150. Preferably, by manual 
control the staples 48 are ejected sequentially at end-diastole 

15 (EKG triggered) one at the time by supplying press gas through a 
supply hose 131. Ejecting one staple simultaneously prepares for 
ejecting the adjacent staple. The pressing gas . is designated by 
arrow Q in Figure 11. The staples 48 should eject sufficiently fast 
in order that the coronary artery wall 23 does not move. Figure 12A 

.20 shows the end result (the staples 48 are drawn schematically). In 
Figure 12B the staples 48 are viewed from above. 

In order to obtain a secure leakfree contact between the 
graft 40 and the artery 23, preferably eight staples 48 are posi- 

25 tioned according to fig. 12B. The staple 48 is about 100 micron 
thick and has a pre-bent form which causes it to behave like a 
paper staple (Figure 12C). Alternatively, staple 48 may have a pre- 
bent form which causes it to turn back on itself, as shown in Fig- 
ure 12D. The staple 48 should pass through a good deal of recipient 

30 wall, i.e. the wall of artery 23, but preferably not enter the 
recipient lumen. The material for staples 48 may be titanium as is 
being used for stents. The staples 48 are non ferromagnetic. 
Instead of staples, other bonding means known by persons skilled in 
the art may be used. 

35 

Once the bypass graft 44 has been anastomosed to the coronary 
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artery 23, the stapler/suturing device 44 is lifted from the ultra- 
sound device, suture 42 is cut and slit 45 is widened sufficiently 
to exit graft 44 through slit 45. Finally, suction on openings 153 
is released and both the stapling/suturing device and the ultra- 
5 sound device are brought outside the chest. 

The ultrasound guidance provided by the US GAD 190 will help 
to avoid highly calcified anastomosis sites, because calcific 
deposits might hamper the staple's course. 

10 

Below, punching a hole in the wall of a coronary artery (dis- 
tal anastomosis) will be described, referring to figures 13 and 14. 

The current Tulleken method to punch a hole in the coronary 
15 artery wall is described in: 

- Tulleken CAF, Verdaasdonk ' RM, Berendsen W, Mali WPTM: Use 
of the excimer laser in high flow bypass surgery of the brain. J 
Neurosurg 1993;78:477-80. 

20 

In the technique according to the prior art for ablating the 
recipient arterial wall 61 (Figure 12A) to establish continuity 
between the graft (donor) lumen 63 and the recipient coronary 
artery lumen 62 use is made of an excimer laser (308 nm) and a 

25 densily packed multif iber catheter which ablates the tissue of the 
recipient wall 61. To avoid creating large water vapor bubbles (3 
mm diameter within 100 microseconds, lifetime 200 microseconds), a 
multifiber catheter may be used which has only a single row of 
fibers at the perimeter of the catheter (not shown). By suction 

30 through its lumen a cut out disk of tissue is retrieved. 

Alternatively, a circular cutter 73 (Fig. 13) or electrical 
spark erosion (Fig. 14) may be employed because these methods are 
cheeper. The latter can be made in a bevelled ellipsoid configur- 
35 ation (not shown) if the graft and recipient artery meet under an 
angle much less than 90°. 
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The circular cutter 73 shown in Figure 13 is fabricated like 
a Simpson atherectomy cutter, as known from: US patent 4,781,186 by 
Simpson JB et al. and from e.g. Johnson DE, Hinohara T, Selmon MR, 
Braden LJ, Simpson JB. Primary peripheral arterial stenoses and 
5 restenoses excised by transluminal atherectomy: a histopathologic 
study. J Am Coll Cardiol 1990;15:419-25. The circular knife 73 in 
various diameters (1.5-3.5 mm) is driven by a straight shaft 72 
which is connected to a flexible cable 70, preferably, made from 
strands of wire which are for 50% counter wound to have maximum 
10 flexibility. In a preferred embodiment, this cable 70 is connected 
to a small battery powered, hand held motor (not shown) which is 
switched on/off with a simple switch. 

The cutter housing 71 is advanced through a thin walled, 
15 flexible, atraumatic catheter 67 which has a carefully polished, 
rounded end. The thin walled catheter 67 has been advanced first to 
minimize injury to the graft vessel wall by the much stiffer circu- 
lar cutter. 

20 A light suction 69 in the thin walled catheter 67 is switched 

on to grab the recipient wall 61 . The circular cutter 73 is rotated 
at about 1000 RPM and advanced like the Simpson cutter, but its 
progress is limited (adjustable) to slightly over the thickness of 
the recipient wall as previously determined by the USGAD 190. When 

25 the circular cutter 73 is retracted, the cut tissue should remain 
in the cutter 73 due to the suction (the cutter housing 71 has a 
few holes, not shown, at its proximal end). 

When electrical spark erosion (Fig. 14) is used, a circular 
30 ring or ellipsoid monopolar electrode 80 isolated laterally by an 
isolator 81 and connected to a catheter lead 82, which is connected 
to spark erosion ablation means 83, is applied. The spark erosion 
method may be accomplished according to the method originally 
described by Slager CJ, Essed CE, Schuurbiers JHC et al., in 
35 "Vaporization of atherosclerotic plaques by spark erosion", J Am 
Coll Cardiol 1985;5:1382-6, and by Slager CJ, Bom N, Serruys PW, 
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Schuurbiers JCH, Vandenbroucke WVA / I,ancee CT, in "Spark erosion 
and its combination with sensing devices for ablation of vascular 
lesions". In: Vogel JHK, King SB, "Interventional Cardiology: 
Future Directions"- St. Louis: The CV. Mosby Company, 19B9, pp. 
5 157-169. Again, suction in the thin walled catheter 67 is employed 
to keep the punched out disk from embolizing into the lumen 62 of 
the recipient artery 23. 

As an alternative to the methods described in Figures 13 and 
10 14, in the aortic wall 135 (proximal anastomosis) one can simply 
punch a hole, as will be described referring to Figure 15. 

A catheter aortic punch has a sharp screw 84 on top of a 
retraction circular knife 85 which enables screwing the screw 84 

15 through the aortic wall 135. By retraction of the central shaft 86 
of the distal circular knife 85 against the advance movement of a 
motor driven proximal circular knife 83 with a fractionally larger 
diameter than the diameter of retraction circular knife 85, a disk 
of for example 3.5 mm is punched out of the aortic wall 135 and 

20 retrieved. 

Again, the thin walled flexible catheter 67 is inserted 
first, through the temporary side branch 132 (Figure 11) of the 
graft 40 to protect the graft wall from mechanical trauma by the 
25 stiff cutting device (see also Figure 13). 

The robotic surgical system described above can also be used 
for open chest CABG surgery on the beating heart. In that case, the 
constraints on seize and positioning of the instruments do not hold 

30 anymore (everything has to pass through 15 mm I.D. trocarts). Fur- 
thermore, the minimally invasive robotic surgical system can also 
be used for other operations, e.g. to remove thoracoscopically a 
myocardial bridge over a coronary artery. Subepicardially located 
accessory atrioventricular pathways may similarly be mapped with 

35 robotic motion compensated electrodes and interrupted surgically 
by this minimally invasive system. 
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In general, virtual object arrest of a moving object within 
the video image frame can be employed as a 2D or 3D diagnostic 
system for close inspection of e.g. a moving part of a machine when 
the machine is preferably not to be stopped. In conjunction with 
5 the diagnostic system , the robotic tracking with repair instruments 
will allow robotic manipulation and repair of the moving part of 
the machine. 
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Claims 

1. An inspection system comprising: 

- at least a first and a second camera to be mounted in a 
5 fixed position relative to each other and to direct their respect- 
ive optical ranges to a moving target; 

an image processor receiving output signals from both the 
first and second cameras, the output signals containing video 
images of the moving target, said image processor being programmed 
10 to 

* detect reference locations of beacons on the moving target 
area in a reference video image, as well as locations of said bea- 
cons in each subsequent video image; 

* compute a set of distances between the locations of said 
15 beacons in each subsequent video image and the corresponding refer- 
ence locations, said set of distances being used to translate and 
rotate said locations of said beacons in each subsequent video 
image in order to match them with said reference locations and to 
obtain a substantially arrested image of the moving target area 

20 (virtual target image arrest); 

- display means connected to the image processor to receive 
and display said arrested image of the moving target area. 

2. An inspection and robotic tracking system comprising: 

25 - at least a first and a second camera to be mounted in a 

fixed position relative to each other and to direct their respect- 
ive optical ranges to a moving target; 

- an image processor receiving output signals from both said 
first and second cameras, said output signals containing video 

30 images of said moving target, said image processor being programmed 
to 

* detect reference locations of beacons on the moving target 
area in a reference video image, as well as locations of said bea- 
cons in each subsequent video image; 

35 * compute a set of distances between the locations of said 

beacons in each subsequent video image and the corresponding refer- 
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ence locations, said set of distances being used to translate and 
rotate said locations of said beacons in each subsequent video 
image in order to match them with said reference locations and to 
obtain a substantially arrested image of the moving target area 
5 (virtual target image arrest); 

- display means connected to the image processor to receive 
and display said arrested image of the moving target area. 

• at least one robot arm to be mounted in a fixed position 
relative to the first and second cameras; 
10 - at least one manipulation instrument to be connected to 

said robot arm and to a robotic computer system to control its 
actions; 

- a tracking control unit connected to the image processor 
and to the robot arm in order to supply the robot arm with control 

15 signals determined by output signals of the image processor; 

at least one control robotic instrument to be manually 
operated and connected to said robotic computer system to supply 
said robotic computer system with control signals. 

20 3. An inspection and robotic tracking system according to 

claim 2, wherein said set of distances being used to translate and 
rotate said locations of said beacons in said subsequent video 
images in order to match them with said reference locations and to 
obtain a substantially arrested image of the moving target area is 

25 converted by the image processor into control signals for the robot 
arm in such a way that, when no human operator operates said con- 
trol robotic instrument, the manipulation instrument connected to 
the robot arm substantially tracks the moving target area, i.e. the 
position of the manipulation instrument relative to the moving 

30 target area remains substantially the same. 

4. A medical system comprising: 

- at least a first and a second endoscopic video camera 
mounted on a first and a second mount, respectively, to be posi- 

35 tioned in a chest cavity in such a way that they both observe a 
predetermined moving target area within the chest cavity; 
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- an image processor receiving output signals from both said 
first and second video cameras , said output signals containing 
video images of said moving target, said image processor being 
programmed to 

5 * detect reference locations of beacons on the moving target 

area in a reference video image, as well as locations of said bea- 
cons in each subsequent video image; 

* compute a set of distances between the locations of said 
beacons in each subsequent video image and the corresponding refer- 

10 ence locations, said set of distances being used to translate and 
rotate said locations of said beacons in each subsequent video 
image in order to match them with said reference locations and to 
obtain a substantially arrested image of the moving target area 
(virtual target image arrest); 

15 - display means being connected to the image processor to 

receive and display said arrested image of the moving target area. 

5. A medical system according to claim 4, wherein the first 
and second video cameras are mounted on adjustable ball bearings on 

20 the first and the second mount, respectively, which mounts, during 
operation, are inserted through holes between neighbouring ribs of 
a chest in order to position the video cameras in the chest cavity. 

6. A medical system according to claim 4, wherein the display 
25 means comprises at least one set of operating spectacles. 

7. A medical system according to claim 4, wherein said bea- 
cons are anatomic locations within the moving target area. 

30 8. A medical system according to claim 4, wherein said bea- 

cons are artificial beacons installed on the moving target area. 

9. A medical system according to claim 6, wherein said spec- 
tacles comprise miniature color LCD's. 

35 

10. A medical system according to claim 4, further comprising 
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a main camera mounted on an adjustable ball bearing to be inserted 
into the chest between neighbouring ribs and to supply video image 
information of any area within the chest to at least one video 
monitor . 

5 

11. A medical system according to claim 4, wherein the image 
processor is connected to a voice activator receiving voice signals 
from a microphone, the voice activator , during operation, generat- 
ing zooming-in or zooming-out commands for zooming- in or zooming- 

10 out the video image of the first and second cameras. 

12. A medical system according to claim 11, wherein zooming- 
in or zooming-out is centred at a point in the video image indi- 
cated by a cursor which may be shifted by voice command. 

15 

13. A medical system according to claim 4, further compris- 
ing: 

- at least one robot arm to be mounted in a fixed position 
relative to the first and second cameras; 

20 - at least one surgical instrument to be connected to said 

robot arm and to a robotic computer system to control its actions; 

- a tracking control unit connected to the image processor 
and to the robot arm in order to supply the robot arm with control 
signals determined by output signals of the image processor; 

25 - at least one control robotic instrument to be manually 

operated and connected to said robotic computer system to supply 
said robotic computer system with control signals. 

14. A medical system according to claim 1 3 , wherein each 
30 robot arm during operation is inserted into the chest cavity 

between two neighbouring ribs. 

15. A medical system according to claim 13, wherein said set 
of distances being used to translate and rotate said locations of 

35 said beacons in said subsequent video images in order to match them 
with said reference locations and to obtain a substantially 
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arrested image of the moving target area is converted by the image 
processor into control signals for the robot arm in such a way 
that, when no human operator operates said control robotic instru- 
ment, the surgical instrument connected to the robot arm substan- 
5 tially tracks the moving target area, i.e. the position of the 
surgical instrument relative to the moving target area remains 
substantially the same. 

16. A medical system according to claim 13 further comprising 
10 ultrasound means to locate and evaluate a coronary artery on a 

beating heart and comprising at least two electromotor driven 
rotating ultrasound transducers (20-60 MHz), the ultrasound means 
being provided with suction means to fix the ultrasound means to 
the beating heart. 

15 

17. A medical system according to claim 16, in which the 
ultrasound means have a hole to receive a robotic stapler /suturing 
device with one end of a bypass graft. 

20 18. A medical system according to claim 17, wherein the 

robotic stapler/suturing device comprises an adjustable expansion 
ring, and a thick walled cylinder provided with an aperture to 
receive the bypass graft to bypass an obstructed part of the coron- 
ary artery and wherein an end part of the bypass graft is to be 

25 expanded in diameter by the expansion ring and folded back around 
the thick walled cylinder by the expansion ring, and wherein the 
thick walled cylinder, during operation, accommodates bonding means 
to bond said end part of the bypass graft to the coronary artery. 

30 19. A medical system according to claim 18, wherein the bond- 

ing means are circumferential staples which have been preformed in 
such a way that during injection both ends of each staple penetrate 
a wall of the bypass graft, enter the coronary artery wall and fold 
toward each other (distal anastomosis). 

35 

20. A medical system according to claim 18, wherein another 
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end of the bypass graft is connected to an aortic wall by preformed 
staples, which penetrate entirely through the wall of the other end 
of the bypass graft, penetrate deeply into the wall of the aorta 
(proximal anastomosis) and folds towards each other. 

5 

21. A medical system according to claim 18 further comprising 
hole making means to be inserted in the bypass graft through a 
temporary side branch after the bypass graft is bonded to the cor- 
onary artery. 

10 

22. A medical system according to claim 21, wherein the hole 
making means comprises a circular cutter within a cutter housing, 
which allows substantially resistance free rotation of the circular 
cutter within the bypass graft. 

15 * 

23. A medical system according to claim 22, wherein the cir- 
cular cutter is rotated by a flexible rotary cable wound from fine 
wires and 50% counterwound to maximize flexibility and keep tor- 
quability, the flexible rotary cable being attached to a hand held, 

20 battery driven motor equipped with a finger switch to activate the 
motor . 

24. A medical system according to claim 22, wherein the cut- 
ter housing contains a channel through which a sub-atmospheric 

25 pressure is applied within the circular cutter. 

25. A medical system according to claim 21, wherein the hole 
making means comprises a static circular or ellipsoid spark erosion 
electrode which is perfectly isolated on the outside, a voltage on 

30 the electrode being supplied by a generator. 

26. A medical system according to claim 21 further comprising 
other hole making means to make a hole in an aortic wall, said 
other hole making means comprising a first circular rotary cutter 

35 driven by a flexible cable and a contra cutter with an exceedingly 
sharp screw front, the contra cutter being housed within the first 
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circular cutter, being driven by another flexible cable housed 
within the first flexible cable, and to be screwed into and through 
the aortic wall to catch a circular piece of aortic wall cut out by 
the first circular cutter. 

5 

***** 



BNSDOCID: <WO 9501757A1 I > 



WO 95/01757 



2/12 



PCT/NL94/00156 



fi g - Ea f i g - EL 





9501757A1 I > 



WO 95/01757 




PCT/NL94/00156 



3/12 



fin -4 





oxicnnrin- ,u/n ocnnK7A -i i - 



WO 95/01757 



PCT/NL94/00156 



4/12 



fiD-S 



120 

J- 



VIDEO 

MOVING? 

TARGET 




122 




12f 

J- 



BEACONS 
REFERENCE 
IMAGE 



125 



BEACONS 
DISPLACEMENT 







12£. 



IMAGE 
TRANSLATION 
AND 
ROTATION 



VIRTUAL IMAGE ARREST 



127 



F 



12$ 



BNSDOCID: <WO 9501757A1J_> 



WO 95/01757 



5/12 



PCT/NL94/00156 




WO 95/01757 MW m3 PCT/NL94/00156 

6/12 




BNSDOCID: <WO 9501757A1 I > 




fig- 10a 




Okicrwm. -\Mr\ ncn 1 



WO 95/01757 



8/12 



fig-lDL 



XC iU US ^ fr-X* 158 «f 



150 



I— Xrf 
2c « 9 




\*59 




W 



PCT/NL94/00156 



/5/ 





fin-ID 




fig-lDd 




fig-10p 




.9501757A1_I_> 




QWCfW^irV *-\Air\ OW17C7A 1 I -s 



WO 95/01757 



10/12 



PCT7NL94/00156 



l>2 



/3. 



fig-lEa 



^5 




fiq-lEn 
—< — ^ 




fiQ-lEL 




BNSDOCID: <WO 9501757A1J_> 



WO 95/01757 



PCT/NL94/00156 



11/12 




OKTI17C7A1 I 



WO 95/01757 mU Wm PCT/NL94/00156 

12/12 



/ 


fig- 


-as 
















B5 




ft 


\* ^y>< v t-d^z ■ 



BNSDOCID: <WO 9501757A1J_> 



ATIONAL SEARCH REPORT 



ronaJ Application No 

PCT/NL 94/00156 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC 6 A61B19/00 G06T7/20 



According to Intern an onaJ Patent Qagficagon (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 

IPC 6 A61B G06T 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practical, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category * 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to el aim No. 


P.Y 


US, A, 5 279 309 (RUSSEL H. TAYLOR ET AL.) 


1,4,8,11 




18 January 1994 




see abstract 






see column 21, line 46 - column 24, line 4 




Y 


US, A, 5 054 045 (JAMES S. WHITING) 1 


1,4,8,11 




October 1991 




see column 1, line 50 - column 2, line 5 





| j Further documents are listed in the continuation of box C. 



0 



Patent family members are listed in annex. 



* Special categories of cited documents : 

"A' document defining the general state of the art which is not 
considered to be of particular relevance 

*E* earlier document but published on or after the international 
filing date 

"L" document which may throw doubts on priority daim(s) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

'O* document referring to an oral disclosure, use, exhibition or 
other means 

"P* document published prior to the international filing date but 
later than the pnority date claimed 



*T" later document published after the international filing date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

*X" document of particular relevance; the claimed invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document is taken alone 

*Y* document of particular relevance; the claimed invention 
cannot be considered to involve an inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the art. 

*&" document member of the same patent family 



Date of the actual completion of the international search 

28 October 1994 


Date of mailing of the international search report 

•T '** 


Name and mailing address of the ISA 

European Patent Office, p.B. 5818 Patentiaan 2 
Nt - 2280 HV Rijswijk 
Td.(-*3l-70) 340-2040, Tx. 31 651 eponl. 
Fax: 31-70) 34O-30I6 


Authorized officer 

Chateau, J-P 



Form PCT/1SA/2I0 (iccond thect) (July 1992) 



^^Kfoi 



NATIONAL SEARCH REPORT 

n formation on patent family members 



aona! Application No 

PCT/NL 94/00156 



Patent document 
cited in search report 



Publication 
date 



Patent family 
member (s) 



Publication 
date 



US-A-5279309 
US-A-5054045 



18-01-94 
01-10-91 



NONE 



EP-A- 
JP-T- 
W0-A- 



0515641 
5504087 
9209184 



02-12-92 
01-07-93 
29-05-92 



Form PCT/1SA/310 (patent ftmUy annex) (July 1993) 

BNSDOCID: <WO_9S01 757A1 I > 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 



Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 



□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 



>d REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 
□ OTHER: 



IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



BEST AVAILABLE IMAGES 




FADED TEXT OR DRAWING 




